Headphones are often worn to privately listen to audio sound of an audio source, video source, or a combination. A user may remove and replace the headphones on his or her head more than once during a given time period. Automatically detecting an unworn headphone, removal or replacement of a headphone on the user's head can be used to control playback of audio and/or conserve power in the headphones.